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Foreword 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  conducts  research  to  support 
Army  personnel  and  training  goals.  In  recognition  of  the  dramatic  changes  Objective  Force  operations  will 
require,  ARI  has  developed  a  research  program  to  identify,  describe,  and  address  future  personnel 
requirements.  This  report  describes  one  aspect  of  the  program. 

The  objective  of  the  “Maximizing  2 1st  Century  Non-Commissioned  Officer  Performance”  (NC02 1 )  research 
project  was  to  improve  21st  century  NCO  performance  by  providing  the  capability  to  identify  and  promote 
highly  qualified  soldiers  with  exceptional  leadership  potential.  The  project  developed,  validated,  and 
evaluated  future-oriented  job  performance  predictor  and  performance  measures.  The  project  was  conducted 
with  support  from  the  Army  Deputy  Chief  of  Staff  for  Personnel  (the  Army  G- 1 )  and  the  Army  Training  and 
Doctrine  Command  (TRADOC).  The  findings  have  been  briefed  to  the  U.S.  Army  Total  Personnel  Command, 
TRADOC,  and  the  Army  G-l.  This  report  provides  a  description  of  the  research  effort  and  discusses  how 
the  research  findings  could  provide  a  resource  for  improving  the  junior  NCO  promotion  system. 
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NCO  Vision 


An  NCO  Corps,  grounded  in  heritage,  values  and  tradition, 
that  embodies  the  warrior  ethos;  values  perpetual  learning; 
and  is  capable  of  leading,  training,  and  motivating  soldiers. 


Effecli\ety  Counsels  4  Mentors  Subordinates 
Maintains  an  Outstanding  Personal  Appearance 
Disciplined  Lenders  Produce  Disciplined  Soldiers 


Jack  L.  Tilley 

12th  Sergeant  Major  of  the  Army 
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The  System 

It  has  over  81,000  ‘members.’  It  moves  30,000  people  from  soldier  to 
sergeant  each  year.  It  requires  the  judgment,  perception,  and  decision 
making  skills  of  the  Company  Commander  and  the  First  Sergeant  in  a 
procedure  that  affects  not  only  individuals’  careers  but  the  future  welfare 
of  the  entire  Army.  It  is  at  once  exciting,  challenging,  rewarding, 
complex,  and  worrisome.  “If  is  the  semi-centralized  promotion  system. 

In  1 998  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI)  started  a  research  project  to  identify  the  characteristics 
of  21st  century  Army  NCOs,  called  NC021.  This  effort  involved  a 
comprehensive  analysis  of  future  conditions  and  job  demands  to  identify 
the  knowledges,  skills,  and  attributes  (KSAs)  that  Objective  Force 
NCOs  will  need.  Over  time,  this  effort  became  focused  on  junior  NCOs 
-  the  movement  of  specialists/corporals  to  sergeants  and  the 
advancement  of  sergeants  to  staff  sergeants. 

The  illustration  on  page  2  outlines  the  scope  of  the  NC021  project. 
The  challenge  was  to  develop  tools  to  measure  relevant  KSAs  along 
with  tools  that  measure  job  performance,  then  to  find  out  if  these 
measures  perform  as  anticipated  by  administering  them  to  soldiers  and 
analyzing  the  data.  If  they  do  work,  they  would  provide  an  improved 
way  of  selecting  junior  leaders.  A  key  element  to  the  project  is  its  future 
orientation  -  the  effort  begins  to  lay  a  foundation  for  NCO  promotion 
as  the  Army  adjusts  to  Objective  Force  operations,  allowing  the  Army 
to  be  proactive  in  planning  for  these  changes.  Transformation  to  the 
Objective  Force  is  not  served  by  relying  on  20th  century  promotion 
recommendation  and  selection  procedures. 

The  development  effort  continues.  This  report  is  designed  to  inform 
and  update  the  user  community  and  others  who  are  involved  in  the 
promotion  of  those  at  the  entry  level  of  the  NCO  pyramid. 
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“  ‘The  only  lesson  of 
history  is  that  men 
never  learn  anything 
from  history,’  said 
George  Hegel.  In  the 
past  several  years 
promotions  to 
sergeant  and  staff 
sergeant  have  risen 
dramatically,  due  in 
part  to  the  Change  In 
NCO  Structure.  Yet 
the  question  arises, 
‘Are  we  filling  the 
ranks  with  unskilled 
soldiers,’  and  if  so, 
are  we  going  to  pay 
for  it  in  future 
conflicts?” 

CSM  Dan  Elder, 
The  NCO  Journal,  Winter  2001 


Field  Validation 
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NC021  Project  Overview 


Define  NCO 
Requirements 


Identify  Objective  Force  NCO  Knowledges, 
Skills,  and  Attributes 

-Review  past  work 

-  Convene  expert  panels  made  up  of  military 
experts  and  psychologists 


Identify  Objective  Force  NCO 
performance  requirements 

-Review  past  work 
-  interview  a  wide  range  of 
Subject  Matter  Experts 


Create/ 

Identify 

Measures 


Predictors 

Existing 

♦  Self-report  Promotion  Point  Worksheet  Administrative  Points 
score  (e.g.,  Awards,  Letters  of  Recognition,  Military 
Education) 

♦Armed  Services  Vocational  Aptitude  Battery  (ASVAB)- 
General  Cognitive  Aptitude 

♦Temperament  Indicator  Scales  (e.g.,  Interpersonal  Skills, 
Work  Orientation,  Adjustment) 

Newly  Created 

♦  Experience  and  Activities  Record  (ExAct)-Adivities  that  may 
predict  performance  at  the  next  grade 

♦  Leadership  Judgment  Exercise  (LeadEx)-Critical  job  and 
supervisory  situations  with  suggested  courses  of  action 

♦  Semi-Structured  Interview-Interviewer  training,  questions, 
and  probes  for  conducting  interviews 


Job  Performance 

♦  Observed  Performance  Rating  Scales 
Supervisor  ratings  on  19  specific 
performance  dimensions,  overall  NCO 
effectiveness,  and  potential  effectiveness 
as  a  Senior  NCO 

♦  Expected  Future  Performance  Rating 
Scales  Supervisor  ratings  on  six 
scenarios  describing  emerging  or 
plausible  future  conditions 


Collect 

Data 


Analyze 


Data 


Soldiers  tested  at  seven  locations 
■449  E4s 
•885  E5s 
•557  E6s 

Total  =  1,891  Soldiers 


Ratings  received  from  988 
supervisors;  coverage  for  over 
70%  of  the  participating  soldiers 


How  it  Works  Now 

The  information  that  currently  factors  into  semi-centralized  promotion 
decisions  is  summarized  in  the  box  below.  Completing  Promotion  Point 
Worksheets  is  currently  a  significant  administrative  burden  because 
most  of  the  information  and  the  process  of  pulling  it  together  are  not 
automated.  The  Army  is  moving  quickly  toward  automation,  which 
will  make  it  easier  to  introduce  changes  to  the  system  and  changes  are 
less  likely  to  present  an  undue  burden  to  the  field. 
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Overview  of  the  Current  Semi-Centralized  Promotion  Evaluation  System 

DA  Form  3355  (Promotion  Point  Worksheet): 

Duty  Performance  Evaluation 

Competence,  Military  Bearing,  Leadership,  Training, 

Responsibility  and  Accountability 

Promotion  Board  Appraisal 

Personal  Appearance,  Bearing,  Self-Confidence 
Oral  Expression  and  Conversational  Skills 
Knowledge  of  World  Affairs 
Awareness  of  Military  Programs 
Knowledge  of  Basic  Soldiering 
Soldier's  Attitude 


Military  Training 

100  points 

APFT,  Weapons  Qualification 
Awards,  Decorations,  Achievements 

—  Administrative 

100  points 

Military  Education 

200  points 

Civilian  Education 

100  points 

800  points 

Other 

Time  in  service  (primary  zone):  to  E5  =  34  months,  to  E6  =  82  months. 

HS  diploma  or  GED. 

Completed  PLDC  or  BNCOC  or  conditional  promotion  pending  completion. 

Eligible  to  reenlist  (waiver  for  AWOL,  courts  martial,  lost  time). 

Medically  qualified. 

Not  flagged,  in  weight  control  program,  or  in  alcohol/drug  abuse  control  program. 

A  minimum  of  350  points  required  for  promotion  to  E5  and  450  points  for  promotion  to  E6. 
Promotions  are  made  by  DA  monthly  based  on  vacancies  existing  in  individual  MOS. 


150  points 


150  points 
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Why  change  the 
promotion  system?  Didn’t 
it  just  change?  Can’t  we 
leave  it  alone  for  a  while? 

The  semi-centralized  promotion 
system  is  never  static.  Change 
occurs  as  the  overall  posture  and 
needs  of  the  Army  change.  Soldier 
evaluation  becomes  more  efficient. 
Job  requirements  and  performance 
predictors  become  more  defined.  In 
the  near  future,  the  entire  semi- 
centralized  promotion  system  will 
become  automated,  allowing  forthe 
incorporation  of  different  promotion 
features  without  additional 
administrative  burden. 

But  the  primary  reason  is  that  the 
Army  is  changing.  The  NCO 
demands  and  requirements  of  the 
Objective  Force  encompass  -  and 
then  surpass  -  the  demands  of  the 
1980s  and  1990s.  Selection  of 
those  who  lead  must  reflect  these 
changes. 

This  does  not  mean  that  all  of  the 
measures  and  findings  outlined  in 
this  report  will  become  part  of  that 
future  promotion  system.  Some 
need  more  work,  some  are  nearer 
term  than  others,  and  all  need 
official  approval  before  they  are 
incorporated.  But  change  to  the 
semi-centralized  promotion  system 
is  inevitable  and  we  are  developing 
specific  ideas  for  change  to 
consider  further. 

(In  that  regard,  please  note  the 
disclaimer  that  the  contents  of  this 
report  are  not  official  Army  policy.) 


Objective  Force  Key  Leader 
Characteristics 

The  analysis  of  Objective  Force  NCO  requirements  involved  two 
objectives: 

y  Developing  a  description  of  forecasted  conditions  affecting 
future  NCO  performance. 

J  Providing  a  descriptive  list  of  the  main  qualities  (i.e., 
KSAs)  needed  for  effective  NCO  performance. 

Note  that  the  effort  targeted  general  NCO  qualities;  not  MOS-specific 
requirements.  We  recognize  the  criticality  of  MOS-specific  skills, 
particularly  as  a  prerequisite  for  promotion,  but  our  purview  was  to 
follow  the  common  soldier  model  currently  used.  Parts  of  specific  MOS 
were  investigated  during  the  analysis  but  only  if  they  were  typical  of 
future  trends  that  all  NCOs  would  need  (e.g.,  digital  operations, 
changing  missions). 

The  analysis  identified  six  primary  characteristics  that  will  affect  all 
Army  personnel  in  the  future. 

Transition  to  digital  operations  and  an  ever  increasing  pace 
of  adaptation  of  technologies.  The  hallmark  of  the  early 
Objective  Force  Army  will  be  the  phased  introduction  of 
information-age  operations  based  on  digital  connection  and 
supporting  software.  All  aspects  of  combat,  combat  support,  and 
combat  service  support  will  be  affected.  Tactical  digital  links 
will  be  crucial,  but  increasingly  non-tactical  applications  will 
likely  utilize  militarized  adaptations  of  commercial  hardware 
and  software.  Change  will  occur  rapidly  -  just  keeping  up  will 
be  a  challenge. 

Diverse  missions  and  frequent  deployments.  While  no  peer 
opponent  is  likely  to  emerge  during  the  first  20  years  of  the 
century,  the  number  and  complexity  of  Army  operations  will 
increase.  Regional  conflicts,  peacekeeping,  peacemaking, 
humanitarian  missions,  support  to  coalition  forces,  anti-terrorism, 
and  domestic  assistance  are  all  likely  to  occur  during  the  era. 
Each  non-traditional  mission  will  be  somewhat  unique  and  there 
will  be  little  carryover  and  lessons-learned  from  preceding 
missions.  Deployments  will  tend  to  be  smaller  in  size,  for  shorter 
periods  of  time  (as  compared  with  20th  century  conflicts),  but 
more  frequent. 

Diversity  of  forces  and  mission-specific  organizations. 

Experimentation,  change,  innovation,  and  inconsistency  will 
mark  the  period.  Some  units  and  soldiers  will  experience  the 
very  latest  in  technology  and  capability  while  others  will  operate 
similarly  to  the  Army  of  the  mid-1990s.  More  and  more  units 


will  fill  niches  in  equipment,  missions,  training,  and  technology 
and  there  will  be  less  standardization  across  the  force,  even 
between  like  units. 

Decentralized  operations.  Technological  advances  will  increase 
the  physical  dispersion  between  elements  and  between  supporting 
and  supported  units.  Employment  of  elements  will  be  less 
hierarchical.  Missions  will  rely  on  smaller,  more  autonomous 
elements.  Operations  will  be  driven  to  lower  levels  with 
companies  and  even  platoons  becoming  the  focal  point  for  many 
operations. 

Changes  in  training  structure,  requirements,  and  delivery. 

Training  will  be  typified  by  an  increasing  reliance  on  technology, 
less  institutional  attendance,  and  an  increased  dependence  on 
self-development  training.  Self-development  training  will 
become  an  increasing  requirement  in  career  learning,  including 
subjects  that  were  formerly  part  of  unit  training  curriculums. 

Changes  in  the  youth  population  and  recruiting  needs.  Army 
active  duty  non-prior  service  accession  requirements  will  be 
around  90,000  annually.  Competition  for  high  quality  youth  (i.e., 
high  school  graduates  in  the  upper  two-thirds  mental  category) 
will  stiffen  as  more  choices  are  available  to  individuals 
considered  in  the  prime  recruiting  market. 

Based  on  this  operational  framework,  we  defined  critical  soldier 
requirements  (i.e.,  KSAs).  This  presented  a  unique  challenge  because 
we  were  projecting  KSAs  to  future  requirements  that  had  not,  in  most 
cases,  been  fully  experienced.  In  the  end,  the  KSAs  are  a  combination 
of  existing,  well-established  requirements  along  with  some  newly 
formed  ones.  All  were  redefined  to  reflect  the  Army  Objective  Force 
characteristics. 

The  38  KSAs  in  the  list  that  starts  on  page  6  were  identified  and  defined. 
These  KSAs  are  consistent  with,  and  supportive  of,  the  doctrine  of 
what  leaders  must  be,  know,  and  do  as  presented  in  FM  22-100,  Army 
Leadership.  Although  not  designed  to  fit  exactly  in  the  FM  22-100 
framework,  the  NC021  KSAs  can  be  roughly  sorted  into  the  required 
leadership  skills  (see  chart  to  the  right).  For  example, 

Conceptual  Skills  include  Adaptability  and  General 
Cognitive  Aptitude.  Under  Interpersonal  Skills  would 
be  the  KSAs  of  Concern  for  Soldier  Quality  of  Life 
and  Team  Leadership. 

Both  current  and  future  NCO  requirements  can  be  sorted 
in  different  ways,  called  by  different  names,  and 
identified  at  different  levels  of  detail.  The  key  is  being 
clear  on  what  is  expected  and  building  NCO  selection, 
training,  development,  and  reward  systems  around  those 
expectations. 
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“As  the  Army 
transforms  to  the 
Objective  Force  we 
must  adjust  our 
personnel  system  in 
anticipation  of  future 
demands.  Our  focus 
will  always  be  on  the 
development  and 
identification  of  a 
professional  NCO 
corps  that  will  meet 
the  full  spectrum  of 
emerging  challenges. 
When  the  promotion 
system  changes,  it 
will  change  because 
we  have  discovered 
proven  ways  to  make 
it  better  and  more 
predictive  of 
leadership 
performance.” 

SGM  Gerald  Purcell, 
Directorate  of  Military 
Personnel  Policy,  Army  G-l 
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Starred  (★)  KSAs  were  judged  by 
subject  matter  expert  panels  as  most 
important  for  promotion  to  E5  and/ 
orE6. 


Sponsored  by... 

This  work  was  directed,  and  is 
being  monitored  by,  the  Directorate 
of  Military  Personnel  Policy 
(DMPP),  Army  G-1.  The  Army  G-1 
is  the  Army’s  Human  Resource 
provider,  responsible  for  programs, 
policies,  and  systems  that  shape 
and  enable  the  manning  and  well¬ 
being  of  the  force.  It  is  responsible 
for  enhancing  the  readiness  and 
institutional  strength  of  the  Army. 
This  work  supports  the  Army 
Transformation  Program. 


NC021  Knowledges,  Skills,  and  Attributes  (KSAs) 


★  Adaptability.  Can  modify  behavior  or  plans  as  necessary  to  reach  goals  or  to  adapt  to 
changing  goals.  Is  able  to  maintain  effectiveness  when  environments,  tasks,  responsibilities, 
or  personnel  change.  Easily  commits  to  learning  new  things  when  the  technology,  mission, 
or  situation  requires  it. 

★  Common  Task  Knowledge  and  Skill.  Possesses  the  necessary  knowledge  and  skill 
to  perform  common  tasks  at  the  appropriate  skill  level  (e.g.,  land  navigation,  field  survival 
techniques,  and  NBC  protection). 

★  Conscientiousness  and  Dependability.  Is  trustworthy,  reliable,  planful,  and 
accountable.  Respects  the  chain  of  command  and  the  value  of  discipline.  Does  not  try  to 
avoid  responsibility. 

★  Directing,  Monitoring,  and  Supervising  Work.  Works  with  subordinates  one-on-one 
to  assign  tasks  and  set  individual  goals  for  work  and  assignments.  Ensures  that 
assignments  are  clearly  understood.  Monitors  individual  subordinate  performance  and 
gives  appropriate  feedback. 

★  Effort  and  Initiative.  Demonstrates  high  effort  in  completing  work.  Takes  independent 
action  when  necessary.  Seeks  out  and  willingly  accepts  responsibility  and  additional 
challenging  assignments.  Persists  in  carrying  out  difficult  assignments  and  responsibilities. 

★  General  Cognitive  Aptitude.  Has  the  overall  capacity  to  understand  and  interpret 
information  that  is  being  presented,  the  ability  to  identify  problems  and  reason  abstractly, 
and  the  capability  to  learn  new  things  quickly  and  efficiently. 

★  Integrity  and  Discipline.  Maintains  high  ethical  standards.  Does  not  succumb  to 
pressure  to  commit  prohibited,  harmful,  or  questionable  acts.  Demonstrates  trustworthiness 
and  exercises  effective  self-control.  Understands  and  accepts  the  basic  values  of  the 
Army  and  acts  accordingly. 

★  MOS/Occupation-Specific  Knowledge  and  Skill.  Possesses  the  necessary  technical 
knowledge  and  skill  to  perform  MOS/occupation-specific  technical  tasks  at  the  appropriate 
skill  level.  Stays  informed  of  the  latest  developments  in  one’s  particular  field. 

★  Motivating,  Leading,  and  Supporting  Subordinates.  Recognizes,  encourages,  and 
rewards  effective  performance  of  individual  subordinates.  Corrects  unacceptable  conduct. 
Communicates  reasons  for  actions  and  listens  effectively  to  subordinates  one-on-one. 
Fosters  loyalty  and  commitment. 

★  Need  for  Achievement.  Has  a  general  but  not  misplaced  confidence  in  own  abilities. 
Seeks  and  enjoys  positions  of  leadership  and  influence.  Is  typically  enthusiastic  and 
energetic.  Sets  high  standards  and  strives  for  accomplishment  and  recognition  in  almost 
any  situation. 

★  Oral  Communication  Skill.  Speaks  in  a  clear,  organized,  and  logical  manner. 
Communicates  detailed  information,  instructions,  or  questions  in  an  efficient  and 
understandable  way.  Note  that  this  skill  refers  to  how  well  the  individual  can  speak  and 
communicate,  not  whether  technical  expertise  is  high  or  low. 

★  Physical  Fitness.  Meets  Army  standards  for  weight,  physical  fitness,  and  strength. 
Maintains  health  and  fitness  to  meet  deployablity  and  field  requirements  as  well  as  the 
physical  demands  of  the  daily  job. 

★  Problem  Solving  and  Decision  Making  Skill.  Reacts  to  new  problem  situations  by 
applying  previous  experience  and  previous  education  and  training  appropriately  and 
effectively.  Does  not  apply  rules  or  strategies  blindly.  Assesses  costs  and  benefits  of 
alternative  solutions  and  makes  timely  decisions  even  with  incomplete  information. 

★  Relating  to  and  Supporting  Peers.  Treats  others  in  the  NCO  corps  in  a  courteous, 
diplomatic,  and  tactful  manner.  Provides  help  and  assistance  to  other  NCOs.  Backs  up 
and  fills  in  for  others  when  needed.  Works  effectively  as  a  team  member. 

★  Team  Leadership.  Communicates  team  goals  and  organizes  and  rewards  effective 
teamwork.  Leads  the  team  to  adapt  quickly  when  missions  change  and  keeps  the  team 
focused  on  new  goals.  Resolves  conflicts  among  team  members.  Shares  relevant 
information  with  team  members. 


U.S.  Army  Research  I 

User  Community  Report 


NC021  Knowledges,  Skills,  and  Attributes  (KSAs)  Continued 


★  Training  Others.  Evaluates  and  identifies  individual  or  unit  training  needs.  Institutes 
formal  or  informal  programs  to  address  training  needs.  Develops  others  by  providing 
appropriate  work  experiences.  Guides  and  assists  subordinates  on  technical  matters. 

Adherence  to  Regulations,  Policies,  and  Procedures.  Adheres  to  policies  and  follows 
prescribed  procedures  in  carrying  out  duties  and  assignments. 

Advanced  Computer  Skills.  Understands  numerous  computer  systems  and  operating 
systems  such  as  Unix,  NT,  and  Army  specific  systems  and  applications.  Can  perform 
routine  troubleshooting  of  computer  systems  and  applications. 

Basic  Electronics  Knowledge.  Knows  general  information  regarding  electronics  principles 
and  electronics  equipment  operation  and  repair.  Knows  general  facts  and  principles  relevant 
for  a  wide  variety  of  electronics  related  tasks,  but  does  not  necessarily  have  highly  specific 
electronics  knowledge  required  for  a  particular  job. 

Basic  Mathematics  Facility.  Knows  and  applies  addition,  subtraction,  multiplication, 
division,  and  simple  mathematical  formulas. 

Basic  Mechanical  Knowledge.  Knows  general  information  regarding  mechanical 
principles,  tools,  and  mechanical  equipment  operation  and  repair.  Knows  general  facts 
and  principles  relevant  for  a  wide  variety  of  tasks  that  require  technical  knowledge,  but 
does  not  necessarily  have  highly  specific  mechanical  knowledge  required  for  a  particular 
job. 

Battlefield  Function  Coordination  and  Management.  Can  individually  apply  and 
effectively  integrate  multiple  battlefield  functions  such  as  direct  and  indirect  fires, 
communications,  intelligence,  and  combat  service  support  to  achieve  tactical  goals. 

Concern  for  Soldier  Quality  of  Life.  Is  aware  of  subordinates’  off-duty  needs  and 
constraints.  Is  sensitive  to  others’  priorities,  interests,  and  values,  and  tries  to  assist 
subordinates  in  making  their  personal  and  family  life  better. 

Cultural  Tolerance.  Demonstrates  tolerance  and  understanding  of  individuals  of  other 
cultural  and  social  backgrounds,  both  in  the  context  of  the  diversity  of  U.S.  Army  personnel 
and  in  interactions  with  foreign  nationals  during  deployments  or  when  training  for 
deployment. 

Emotional  Stability.  Has  the  tendency  to  act  rationally,  to  display  a  generally  calm  and 
even  mood.  Generally  maintains  composure  and  does  not  become  overly  distraught  by 
stressful  situations. 

General  Self-Management  Skill.  Uses  appropriate  strategies  to  self-manage  the  full  range 
of  one’s  work  and  non-work  responsibilities  (e.g.,  assignments,  personal  finances,  family). 
Such  strategies  include  setting  both  long-and  short-term  goals,  prioritizing  goals,  allocation 
of  effort  and  personal  resources  to  goal  priorities,  and  assessing  one’s  own  performance. 
Works  effectively  without  direct  supervision,  but  seeks  help  and  advice  from  others  when 
appropriate. 

Information  Management.  Effectively  monitors,  interprets,  and  redistributes  digital  display 
information  (as  well  as  printed  and  orally  delivered  information)  from  multiple  sources  to 
multiple  recipients.  Sorts,  classifies,  combines,  excludes,  and  presents  information  so 
that  it  is  usable  by  others.  Does  not  readily  succumb  to  information  overload. 

Knowledge  of  Inter-Relatedness  of  Units.  Is  capable  of  analyzing  how  goals  and 
operations  of  own  unit  are  interrelated  with  other  units  and  systems,  and  how  one  unit’s 
actions  affect  the  performance  of  other  units.  Can  see  the  larger  strategic  picture  and 
interpret  how  one’s  own  unit  relates  to  it. 

Military  Presence.  Presents  a  positive  and  professional  image  of  self  and  the  Army  even 
when  off  duty.  Maintains  proper  military  appearance. 

Models  Effective  Performance.  Acts  in  ways  that  consistently  serves  as  a  model  for 
what  effective  performance  should  be  like,  be  it  technical  performance,  military  bearing, 
commitment  to  the  Army,  support  for  the  Army  mission,  or  performance  under  stressful  or 
adverse  conditions.  Can  consistently  set  an  example  for  others  to  follow. 


And,  we  are.... 

The  U.S.  Army  Research  Institute 
\  for  the  Behavioral  and  Social 
;  Sciences  (ARI)  is  part  of  the  G-1 
function  of  the  Army. 
Headquartered  in  Alexandria, 
t  Virginia,  ARI  serves  the  Army 
world-wide  with  nine  field  locations 
and  a  presence  wherever  and 
whenever  there  is  a  personnel 
issue  to  be  addressed.  ARI’s 
mission  is  to  maximize  individual 
f  and  unit  performance  and 
readiness  to  meet  the  full  range  of 
worldwide  Army  missions  through 
advances  in  the  behavioral  and 
|  social  sciences.  It  is  the  primary 
arm  of  the  Army  conducting 
■  research  and  analysis  on 
>  personnel  performance.  ARI’s 
programs  support  three  of  the 
Army’s  Imperatives:  Quality 
People,  Leader  Development,  and 
Training.  Learn  more  about  ARI  at 
http://www.ari.armv.mil. 

The  Human  Resources  Research 
Organization  (HumRRO)  is  a 
private,  non-profit  research  firm 
with  over  50  years  of  experience 
and  dedication  to  working  with  the 
?'  Army.  It  is  staffed  with  scientists 
and  specialists  in  the  areas  of 
military  policy  and  functions, 
personnel  planning  and  policy 
analysis,  and  performance 
measurement  and  assessment.  It 
is  also  located  in  Alexandria, 
Virginia  with  a  number  of  field 
locations.  Find  out  more  about 
I.  HumRRO  at  http://www.humrro.org. 

:  Together,  the  ARI-HumRRO  team 
(  takes  pride  in  its  approach  to 
r  researching  soldier  issues.  The 
focus  is  always  service  to  the  Army. 
I  We  involve  soldiers  in  our  projects 
:  through  panels,  surveys,  interviews, 
j  field  data  collection,  and  product 
reviews. 
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NC021  Knowledges,  Skills,  and  Attributes  (KSAs)  Continued 


Perceptual  Speed  and  Accuracy.  Has  the  ability  to  recognize  and  interpret  visual 
information  quickly  and  accurately,  particularly  with  regard  to  comparing  similarities  and 
differences  among  words,  numbers,  objects,  or  patterns,  when  presented  simultaneously 
or  one  after  the  other. 

Psychomotor  Aptitude.  Has  the  ability  to  coordinate  the  simultaneous  movements  of 
one’s  limbs  (arms,  legs),  to  operate  single  controls  or  to  operate  multiple  controls 
simultaneously,  and  to  make  precise  control  adjustments  that  involve  eye-hand 
coordination. 

Safety  Consciousness.  Follows  safety  guidelines  and  instructions.  Checks  the  behavior 
of  others  to  ensure  compliance. 

Self-Directed  Learning  Skill.  Has  a  clear  goal  of  maintaining  continuous  learning  and 
training  over  one’s  entire  career.  Is  proficient  at  determining  personal  training  needs, 
planning  education  and  training  experiences  to  meet  them,  and  evaluating  one’s  own 
training  success.  Uses  efficient  personal  learning  strategies  (e.g.,  identifying  specific 
learning  goals,  organizing  the  material  to  be  learned,  and  practicing  the  new  skills  in  an 
appropriate  context). 

Selfless  Service  Orientation.  Commits  to  the  greater  good  of  the  team  or  group.  Puts 
organizational  goals  ahead  of  individual  goals  as  required. 

Spatial  Relations  Aptitude.  Has  the  ability  to  mentally  visualize  the  relative  positions  of 
objects  in  two  dimensional  or  three-dimensional  space,  and  how  they  will  be  positioned  if 
they  are  moved  or  rotated  in  different  ways. 

Working  Memory.  Has  the  ability  to  maintain  information  in  memory  for  short  periods  of 
time  and  to  retrieve  it  accurately. 

Writing  Skill,  Communicates  thoughts,  ideas,  and  information  successfully  to  others 
through  writing.  Uses  proper  sentence  structure  including  grammar,  spelling,  capitalization, 
and  punctuation. 


The  Promotion  Predictor 
Challenge: 

Promotion  is  not  a  reward  for  past 
performance  -  it  is  an  expectation 
of  future  performance.  But  it  is  also 
true  that  the  best  predictor  of  future 
performance  is  past  performance. 
Therefore  the  NC021  KSA  list 
includes  job  performance 
requirements  that  are  applicable  (to 
varying  degrees)  to  performance  at 
grades  E4,  E5,  and  E6.  This  allows 
us  to  consider  using  current 
performance  as  a  specialist  (for 
example)  to  help  predict  future 
performance  as  a  sergeant.  The 
more  time  in  service,  the  more 
historical  information  there  is  to 
evaluate,  but  there  will  always  be 
an  amount  of  uncertainty  involved. 
Reducing  that  uncertainty  improves 
the  promotion  system. 


Building  and  Evaluating 
New  System  Components 

The  identification  of  important  NCO  KSAs  was  only  the  first  step. 
Identifying  KSAs  has  little  practical  application  unless  we  measure 
them  so  they  can  be  used  for  making  decisions.  Measurement  presents 
its  own  challenges.  There  are  no  perfect  ways  to  measure  characteristics 
of  people  that,  unlike  height  and  weight,  are  not  directly  observable. 
Our  goal  is  to  make  the  measurement  of  soldiers’  capabilities  as  accurate 
and  reliable  as  possible  -  removing  irrelevant  factors  and  making  the 
selection  fair  and  objective.  This  usually  involves  standardizing  how 
soldiers  are  evaluated. 

We  developed  several  potential  measures  that  could  be  used  as  a  basis 
for  making  decisions  about  promotion.  These  became  our  predictor 
measures  because  we  use  them  to  predict  NCO  performance  at  the 
next  pay  grade.  We  also  developed  two  separate  measures  of  NCO  job 
performance  -  one  for  current  performance  and  another  for  performance 
under  future  conditions.  These  measures  were  the  criteria  against  which 
the  predictor  measures  were  evaluated.  Both  the  predictor  and  job 
performance  measures  went  through  extensive,  painstaking,  and  time 
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consuming  development  procedures.  Then  the  predictor  measures  were 
administered  to  hundreds  of  soldiers  and  performance  ratings  were 
obtained  from  their  supervisors  in  a  field  validation. 

Our  research  focused  on  the  predictability  of  a  soldier’s  performance  in 
the  next  highest  grade  level  as  determined  by  the  relationship  between 
the  KSA  (predictor)  measures  and  the  supervisor  ratings.  Rather  than  track 
soldiers  over  time,  we  administered  the  predictor  and  job  performance 
measures  to  sergeants  and  staff  sergeants.  Looking  at  the  relationship 
between  predictor  and  job  performance  scores  for  the  E5  soldiers  tells  us 
what  is  likely  to  work  best  for  promoting  specialists/corporals  to  sergeant 
and  looking  at  the  same  data  for  the  E6  soldiers  tells  us  what  predictors 
will  likely  work  best  for  promoting  sergeants  to  staff  sergeants.  If  a 
predictor  measure  has  a  high  correlation  with  supervisors’  job  performance 
ratings,  that  means  it  could  be  useful  for  predicting  performance.  An  even 
better  outcome  would  be  if  results  for  the  experimental  predictor  tools, 
combined  with  the  current  Promotion  Point  Worksheet,  show  a 
significantly  higher  correlation  with  the  job  performance  ratings  than  does 
the  current  system  by  itself.  Adding  such  tools  to  the  current  system  would 
improve  the  accuracy  of  promotion  decisions. 

Tools  to  Measure  Job 
Performance 

Within  the  military,  supervisor  ratings  have  been  well  established  as  an 
accurate  means  of  evaluating  both  performance  and  potential  of 
subordinate  soldiers.  It  is  a  comfortable  role  for  most  supervisors.  Two 
different  supervisor  rating  instruments  were  developed:  (a)  one  set  to 
evaluate  performance  in  the  current  Army,  and  (b)  another  set  to  anticipate 
how  soldiers  will  perform  under  expected  future  Army  conditions. 

H  Observed  Performance  Rating  Scales 

These  scales  were  designed  to  provide  an  overall  picture  of  the  individual’s 
current  performance,  plus  potential  as  a  senior  (E7-E9)  NCO.  NCO 
performance  is  multi-dimensional  and  we  identified  1 9  areas  of  functional 


Sample  Rating  Area:  Relating  to  and  Supporting  Peers 

How  effectively  does  this  soldier  relate  to  and  support  peers? 

Tends  to  be  rude,  selfish,  and 
insensitive  to  peers'  concerns; 
generally  fails  to  provide 
assistance  to  others,  even  when 
there  is  a  clear  need  to  do  so; 
may  force  his/her  approach  to 
tasks  on  others  without  seeking 
input. 

Usually  courteous  and  tactful 
when  dealing  with  peers; 
provides  assistance  to  others, 
especially  when  It  is  dear  that 
help  Is  needed;  tries  to  develop 
approaches  to  tasks  that  take 
into  account  obvious  differences 
of  opinion. 

Always  treats  peers  in  a 
courteous  and  tactful  manner; 
offers  assistance  without  waiting 
to  be  asked,  even  In  situations 
that  involve  complicated 
interpersonal  situations;  actively 
seeks  out  peers'  opinions  and 
Incorporates  peers'  ideas  into 
own  plans. 

LOW 

MODERATE 

HIGH 

1  2 

3  4  5 

6  7 
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requirements.  A  sample  scale  is  shown  on  page  9  and  the  complete  list 
of  areas  is  shown  in  the  box  below.  Supervisors  provide  scaled  ratings 
based  on  what  they  have  observed  about  their  soldiers’  performance. 
This  provides  a  complete  description  of  the  individual,  identifying  both 
strong  and  weak  areas.  The  results  are  used  as  one  set  of  criteria  to 
evaluate  the  predictor  measures. 

m  Expected  Future  Performance  Rating  Scales 

It  was  necessary  to  develop  an  instrument  that  focused  on  performance 
under  future  conditions  because  the  focus  of  the  work  is  to  improve 
NCO  promotion  in  the  future  Army.  This  instrument  was  based  on  the 
analysis  of  emerging  Objective  Force  requirements  and  conditions, 
largely  reflected  in  the  six  characteristics  described  on  pages  4-5. 
Collectively,  these  scales  provide  a  picture  of  how  supervisors  predict 
soldiers  will  perform  in  this  future  environment.  Taken  together  with 
the  Observed  Performance  Rating  Scales,  these  scales  provide  the 
complete  set  of  criteria  for  evaluating  the  predictor  measures. 


Observed  Performance  Rating  Scales  -  NCO  Areas  of 
Performance 

■  MOS/Occupation-Specific  Knowledge  and  Skill 

■  Common  Task  Knowledge  and  Skill 

■  Computer  Skills 

■  Writing  Skill 

■  Oral  Communication  Skill 

■  Level  of  Effort  and  Initiative  on  the  Job 

■  Adaptability 

■  Self-Management  and  Self-Directed  Learning  Skill 

■  Demonstrated  Integrity,  Discipline,  and  Adherence  to 
Army  Procedures 

■  Acting  as  a  Role  Model 

■  Relating  to  and  Supporting  Peers 

■  Cultural  Tolerance 

■  Selfless  Service  orientation 

■  Leadership  Skills 

■  Concern  for  Soldier  Quality  of  Life 

■  Training  Others 

■  Coordination  of  Multiple  Units  and  Battlefield  Functions 

■  Problem  Solving/Decision  Making  Skill 

■  Information  Management 

For  a  complete  set  of  the  Observed  Performance  Rating  Scales 
and  other  selected  NC021  measures,  please  submit  a  request  to 
NC021@ari.army.mil. 


Tools  to  Predict 
Performance  in 
the  Next  Grade 

Predictor  measures  were  developed  to 
assess  whether  a  soldier  possesses  the 
relevant  KSAs  for  promotion  to  a  higher 
level  of  leadership.  The  predictor 
measures  were  designed  to  be  suitable  for 
potential  inclusion  in  the  Army’s  semi- 
centralized  NCO  promotion  system, 
which  means  we  had  to  consider  the 
realities  of  operational  administration. 
The  goal  is  to  provide  meaningful 
information  for  making  promotion 
decisions  from  the  grades  of  E4  to  E5  and 
from  E5  to  E6. 

H  The  Personnel  File  Self-Report 

Description:  The  Personnel  File  Self- 
Report  recreates  the  administrative 
information  found  on  the  Promotion  Point 
Worksheet.  During  the  field  validation  it 
was  not  possible  to  access  personnel 
records  and  extract  Promotion  Point 
information.  However,  it  was  necessary 
to  analyze  the  administrative  point  areas 
currently  used  -  this,  in  effect,  was 
baseline  information  reflecting  the 
promotion  status  quo. 
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Previous  work  has  indicated  that  soldiers  are  generally  pretty  accurate 
in  providing  information  about  themselves.  We  therefore  asked  soldiers 
for  the  information  normally  found  on  the  Promotion  Point  Worksheet. 
The  self-report  form  was  not  meant  to  mirror  exactly  the  Promotion 
Point  Worksheet,  only  approximate  it.  The  Personnel  File  Self-Report 
information  was  sufficiently  detailed  so  that  we  were  able  to  simulate 
an  Administrative  Points  score  for  each  soldier. 

Field  Validation  Results:  The  simulated  Administrative  Points  promotion 
score  predicted  performance  fairly  well.  We  found  that  some  of  the 
administrative  categories  predicted  how  soldiers  would  perform  as  a 
sergeant  better  than  how  they  would  perform  as  a  staff  sergeant,  and 
vice  versa.  We  also  looked  at  these  areas  by  job  categories  (clusters  of 
MOS)  and  found  some  differences.  While  we  must  be  cautious  in  this 
regard  because  our  sample  did  not  include  all  MOS  or  represent  all 
career  management  fields,  there  is  evidence  that  prediction  could  be 
improved  by  weighting  the  administrative  categories  differently  by  pay 
grade  and  MOS  or  MOS  cluster.  Indeed,  the  experimental  predictor 
measures  described  below  all  showed  differences  in  how  they  worked 
by  pay  grade  and  MOS  type. 

H  Armed  Services  Vocational  Aptitude  Battery  (ASVAB) 

Description:  The  ASVAB  is  an  existing  measure  used  for  both  selection 
into  the  Army  and  for  MOS  classification.  Additionally,  ASVAB  scores 
are  used  after  a  soldier  is  in  service  to  determine  reenlistment,  and 
sometimes,  schooling  and  duty  options.  Soldiers  can  retake  the  ASVAB 
while  in  service  -  then  it  is  called  the  Armed  Forces  Classification  Test 
(AFCT). 

The  ASVAB  has  several  subtests  that  are  combined  to  create  various 
composite  scores.  One  composite  -  the  Armed  Forces  Qualification 
Test  (AFQT)  -  is  used  to  identify  high  quality  soldiers  for  selection 
into  the  Army.  Nine  composites  are  used  as  prerequisites  for 
classification  into  specific  Army  jobs.  The  11th  composite  -  General 
Technical  (GT)  -  is  used  extensively  in-service  for  schooling  and 
reenlistment  options.  The  ASVAB  provides  a  direct  measure  of  the 
soldier’s  general  cognitive  aptitude  which  is  an  underlying  requirement 
to  many  aspects  of  NCO  performance  such  as  problem  solving  and 
decision  making. 

Field  Validation  Results:  The  ASVAB  showed  some  capacity  to  improve 
the  prediction  for  both  E5  and  for  E6  performance  over  the  current 
system.  In  other  words,  it  would  improve  slightly  over  what  is  currently 
being  used  in  the  Promotion  Point  Worksheet  Administrative  Points 
areas. 

H  Temperament  Indicator  Scales 

Description:  These  scales  provide  measures  of  a  variety  of  life  skills, 
reactions  to  life  events,  behaviors,  and  psychological  constructs.  They 
are  directly  related  to  such  attributes  as  adaptability,  dependability, 
and  interpersonal  skills.  There  is  a  strong  leadership  component  to  these 
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Faking  It? 

A  concern  in  measure  development 
is  that  there  are  certain  instruments 
where  soldiers,  if  they  have  a  stake 
in  the  outcome  (such  as  promotion), 
will  give  the  input  they  think  the 
Army  expects  to  hear,  not 
necessarily  the  most  truthful 
response.  Some  of  the  trial 
predictor  measures  are  more 
susceptible  to  this  than  others.  For 
example,  in  some  measures  (the 
ASVAB  for  example)  there  is  a 
“right’’  answer,  so  faking  is  not  an 
issue  (though  cheating  can  be).  But 
in  other  measures,  the  answer  is  not 
right  or  wrong  but  the  answer  tells 
us  something  about  the  person.  If 
people  are  asked  how  much  they 
cheat  on  their  income  tax,  they  may 
be  less  than  truthful  if  the  response 
can  be  identified  with  them.  In  that 
example,  people  can  tell  the  “good” 
answer  from  the  “bad”  answer  and 
can  vary  their  response  based  on 
what  they  feel  the  effect  of  providing 
a  completely  truthful  response 
might  be. 

There  are  ways  in  which  the 
instruments  can  be  designed  to  help 
guard  against  the  inclination.  For 
example,  some  of  the  Temperament 
Indicator  scales  use  a  so-called 
“forced  choice”  strategy.  There  are 
also  procedures  that  can  be  used 
to  identify  respondents  who  are 
probably  faking. 

Concern  about  this  issue  is  a 
serious  requirement.  It  is  essential 
that  a  trial  validation  period  under 
operational  conditions  be  part  of  the 
development.  The  Army  (its  soldiers 
and  leaders)  must  have  confidence 
that  potential  promotion  instruments 
are  fair,  accurate,  and  reliable. 


Sample  Temperament  Indicator  Items 

1 .  On  a  5-point  scale,  indicate  how  much  you  agree  or  disagree  with  the  following 
statements. 

A. To  what  extent  have  you  enjoyed  thinking  about  the  plusses  and  minuses  of 
alternative  approaches  to  a  problem? 

B.  How  likely  are  you  to  criticize  others  when  they  start  criticizing  you? 

2.  Mark  the  statement  that  is  most  like  you  with  an  "M”  and  mark  the  statement  that 
is  least  like  you  with  an  1.  ” 

_M_A.  I  rarely  got  into  trouble  with  my  high  school  teachers. 


L  B.  I  have  not  exercised  routinely  over  the  past  year. 

_ C.  I  have  enjoyed  being  in  positions  of  authority. 

_ D.  I  sometimes  lose  sleep  over  worries. 


scales  that  has  been  established  in  a  variety  of  other  Army  applications. 
Results  show  that  these  scales  can  be  used  to  predict  far  ranging 
behaviors  from  individual  physical  fitness  to  personal  integrity. 

Field  Validation  Results:  Some  of  the  individual  behavioral  areas  assessed 
by  these  scales  suggest  that  they  could  substantially  improve  the 
prediction  of  performance  over  the  current  Administrative  Points.  For 
example,  Social  Maturity  and  Tolerance  for  Ambiguity  scores  showed 
promise  for  predicting  performance  for  sergeants  whereas  the 
Adjustment  score  was  particularly  useful  for  predicting  staff  sergeant 
job  performance. 

S  Experience  and  Activities  Record  (ExAct) 

Description:  The  ExAct  assesses  how  often  a  soldier  has  engaged  in 
specific  activities  or  experiences  that  may  predict  performance  at  the 
next  grade.  The  ExAct  elicits  verifiable  information  about  individual 
experiences  that  are  often  considered  by  supervisors  when  evaluating 
personnel  for  promotion.  It  is  not  meant  to  capture  normal  duty  or  MOS 
performance  but  to  identify  those  experiences  and  activities  that  occur 
beyond  the  expected  performance  for  the  job  and  grade.  It  codifies 
what  is  already  being  done  on  an  informal  basis. 

Field  Validation  Results:  The  ExAct  did  a  fairly  good  job  at  predicting 
performance  for  sergeants.  However,  we  found  it  provided  only  limited 
improvement  to  the  prediction  of  performance  over  the  Administrative 
Points  portion  of  the  current  Promotion  Point  Worksheet. 

Sf  The  Leadership  Judgment  Exercise  (LeadEx) 

Description:  The  LeadEx  assesses  the  effectiveness  of  judgments  about 
appropriate  courses  of  action  in  various  job-related,  supervisory-based 
scenarios.  Situations  address  areas  of  problem  solving  and  decision 
making,  team  leadership,  dealing  with  subordinates,  and  conducting 
training. 
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Although  the  LeadEx  items 
look  like  traditional  multiple- 
choice  test  questions,  all 
responses  have  some  degree  of 
acceptability;  that  is,  there  is 
not  a  single  “correct”  answer 
with  all  other  choices  being 
incorrect.  Because  each 
possible  response  has  been 
rated  by  senior  NCOs  as  to  its 
effectiveness  for  handling  the 
problem  presented,  the  scoring 
system  allows  some  fairly 
precise  rankings  of  supervisory 
effectiveness. 

Field  Validation  Results:  The 
LeadEx  showed  itself  to  be  a 
promising  instrument, 
particularly  for  predicting 
performance  for  sergeants.  It 
significantly  improved  the 
prediction  of  performance  over 
and  above  current  promotion 
requirements. 

■  Semi-Structured  Interview 


Sample  LeadEx  Items 

Instructions:  For  each  item,  mark  which  course  of  action  you  would  be  most  likely  to  follow 
with  an  "M"  and  mark  the  choice  that  you  would  least  choose  with  an  "L. " 

As  a  junior  NCO,  you  need  to  counsel  a  subordinate.  What  is  the  most  important  thing  to  do 
when  preparing  for  and  conducting  the  counseling? 

_ a.  Prepare  a  course  of  action  that  you  want  the  soldier  to  follow. 

_ b.  Plan  to  guide  and  encourage  the  soldier  to  arrive  at  his  own  solutions. 

in  c.  Focus  on  the  sanctions  and  rewards  that  you  control. 

M  d.  Follow  the  outline  of  the  DA  Form  4856-R,  General  Counseling  Form. 

Your  unit  is  on  a  joint  British-US  exercise.  You  are  supervising  a  joint  communications  center. 
One  of  your  soldiers  is  talking  with  a  British  officer  on  the  phone.  He  greets  him  and  says 
goodbye  using  a  British  slang  term  he  has  heard.  After  he  gets  off  the  phone,  one  of  the 
British  soldiers  in  the  room  informs  your  soldier  that  the  slang  term  he  used  is  considered  an 
insult  in  this  context.  What  should  you  do? 

_ a.  Tell  the  soldier  to  not  use  slang  terms  unless  he  is  sure  of  the  meaning. 

^  b.  Inform  all  staff  members  that  all  business  will  be  conducted  in  a  professional,  military 
manner  no  slang,  street  talk,  etc.  Strictly  military. 

il  c.  Let  the  soldier  involved  and  the  British  soldier  who  corrected  him  work  it  out.  No 
action  is  required  on  your  part. 

_ d.  Explain  to  the  British  soldiers  that  no  insult  was  intended  and  that  you  all  have  to 

learn  about  each  other's  ways  and  tolerate  each  other  while  the  learning  is  going  on. 


Description:  The  semi-structured  interview  was  designed  to  provide  a 
board  evaluation  but  under  more  controlled  conditions  than  exists  with 
the  current  promotion  board  process.  The  interview  is  structured  to 
measure  specific  leadership  traits.  It  provides  for  questions  with  in- 
depth  response  requirements  based  both  on  the  individual’s  past 
experiences  and  on 


hypothetical  situations. 
Soldiers  are  evaluated  using 
a  structured  rating 
procedure. The  interview 
provides  standardized 
measurement  of  adaptability, 
effort  and  initiative,  self¬ 
management  and  self- 
directed  learning,  and 
leadership  potential. 

Field  Validation  Results:  During 
the  field  validation,  the  semi- 
structured  interview  showed 
a  strong  validity  for 
predicting  performance  for 


Sample  Semi-Structured  Interview  Questions 

Past  experiences  in  specific  situations 

Tell  me  about  a  time  when  you  planned  an  event  or  prepared  an  assignment  well  in 
advance  and  your  supervisor  changed  it  at  the  last  minute.  What  did  you  do? 

Hypothetical  situations 

You  have  just  broken  an  important  piece  of  equipment.  If  you  turn  it  in  to  the  armorer,  there 
is  a  chance  it  will  not  be  blamed  on  you.  What  would  you  do? 

"[Conducting  these  interviews]  was  a  real  good  experience.  I  could  see  that 
soldiers  were  really  involved  and  thinking  about  their  answers.  I  saw  some 
real  display  of  skills  and  I  saw  soldiers  who  really  lack  skills.  It  made  me 
reflect  on  my  own  soldiers  and  where  they  would  be  in  such  a  process  and 
what  I  need  to  do  to  improve  them. " 

Comment  from  a  Platoon  Sergeant,  Fort  Stewart  after  sitting  as  a 
Semi-Structured  Interview  Board  Member  for  3  days  during  the  Field  Validation 
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E5  soldiers.  Because  of  time  constraints,  the  interview  was  not 
administered  to  the  E6  soldiers  so  no  conclusions  were  reached  about 
this  group. 

Summary  of  the  Findings 

Validation  is  finding  out  what  works,  what  doesn’t  work,  what  works  a 
little  bit,  and  what  works  a  lot.  In  this  case,  with  a  lot  of  different 
measures  and  a  lot  of  different  groups  (E4s,  E5s,  E6s),  we  need  to  be 
cautious  about  what  we  conclude.  Study,  analysis,  and  review  of  the 
data  that  we  gathered  will  continue,  but  we  do  have  some  preliminary 
indications  about  the  current  promotion  system  and  what  effect  changes 
would  have. 

Just  how  much  improvement  is  achievable  over  the  current  system 
cannot  yet  be  precisely  quantified.  As  indicated  in  our  discussion  of 
each  individual  measure,  however,  most  of  the  predictors  that  we 
examined  showed  potential  as  additions  for  enhancing  the  Army’s  semi- 
centralized  promotion  system.  In  most  cases,  the  results  were  stronger 
when  considering  promotion  to  E5  than  when  applied  to  E6  promotion. 

The  figures  on  pages  14-15  indicate  an  approximate  improvement  if 
all  of  the  experimental  predictor  measures  developed  in  the  NC021 
project  were  added  to  the  existing  Promotion  Point  Worksheet1. 

Taken  together,  the  findings 
indicate  that  the  semi-centralized 
promotion  system  could  be 
improved  and  that  elements  of 
the  measures  we  studied  are  good 
candidates  to  be  part  of  that 
improvement.  There  is  also  some 
compelling  evidence  that  there 
are  differences  between  how 
various  types  of  information  can 
best  be  used  to  forecast 
performance  at  the  sergeant  and 
staff  sergeant  levels.  More 
accurate  promotion  decisions 
would  be  possible  if  these 
differences  were  reflected  in  the 
system  (e.g.,  by  varying  the 
number  of  points  assigned  to 
different  pieces  of  information 
on  the  Promotion  Point 
Worksheet,  depending  on 
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!  Validity  estimates  are  based  on  correlations  between  the  predictors  and  criterion  .  These 
values  can  vary  from  0  to  /,  where  0  indicates  no  association  between  predictors  and  criterion, 
and  1  indicates  a  perfect  association.  Thus,  the  higher  the  values,  the  better  the  prediction. 
These  values  are  based  on  estimates  that  have  been  corrected  for  standard  statistical  artifacts 
(i.e.,  criterion  unreliability,  range  restriction  in  the  predictor,  and  shrinkage). 


/■  • 


U.S.  Army  Research  I 

User  Community  Report 


whether  one  is  up  for  an  E5  or 
E6  promotion).  This  would  be 
easy  to  implement  in  an 
automated  promotion  system. 
Similar  differences  can  be 
identified  by  types  of  jobs 
(MOS  or  clusters  of  MOS)  but 
the  field  validation  is  not  broad 
enough  to  draw  more  than 
preliminary  indications  that 
there  might  be  some  effect  that 
would  warrant  different 
promotion  treatment  by  MOS. 

Other 

Possible 

Applications 
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During  the  field  validation,  a  lot  of  reaction,  some  pro  and  of  course, 
some  con,  to  the  different  measures  was  provided  by  the  participants 
and  supervisors.  However  there  were  three  that  consistently  generated 
the  most  interest  for  potential  uses  other  than  the  semi-centralized 
promotion  structure. 

Observed  Performance  Rating  Scales:  These  were  seen  as  a 
valuable  counseling  tool  by  many  of  the  supervisors.  Mentioned  most 
frequently  as  a  feature  was  the  behavioral  anchor  descriptions  that  could 
be  used  (as  shown  in  the  example  on  page  9),  not  only  to  define  current 
levels  of  performance  but  also  to  pinpoint  what  is  needed  for  individuals 
to  get  to  higher  levels.  Participants  pointed  out  that  the  scales  were  a 
handy  working  document  that  could  be  shared  with  subordinates. 
Several  supervisors  highlighted  the  flexibility  of  the  scales  -  that  they 
could  be  tailored  for  each 
individual,  concentrating  on  just 
a  few  of  the  areas  at  a  time.  To  get 
a  broader  perspective,  ratings 
could  be  collected  from  peers  and 
subordinates,  as  well  as 
supervisors. 

Experiences  and  Activities 
Record  (ExAct):  This  too  was 
seen  as  a  counseling  tool  but 
primarily  as  a  roadmap  for  those 
seeking  to  improve  themselves. 

Several  supervisors  thought  of  it 
as  being  particularly  helpful  to  use 
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NCOES  Applications... 

It  is  not  necessary  for  individual 
measures  that  are  proven 
predictors  to  be  integrated  into  the 
semi-centralized  promotion  system 
to  have  an  effect  on  future 
leadership.  Many  of  the  measures 
can  be  used  as  counseling  tools  to 
change  or  guide  behaviors,  not  just 
to  measure  KSAs.  The  path  to  NCO 
leadership  is  one  of  learning  and 
experience  in  a  great  number  of 
areas.  Many  of  these  measures  can 
provide  an  individualized  roadmap 
for  soldiers. 


with  their  “fast  track”  specialists  and  corporals.  Not  everyone  agreed 
completely  with  all  of  experiences  and  activities  listed  and  some  felt 
they  would  modify  or  delete  some  of  the  items.  Foremost  among  the 
changes  was  to  add  blank  lines  to  list  MOS-  or  unit-specific  experiences 
and  activities. 

Semi-Structured  Interview:  Most  of  the  reaction  came  from  senior 
NCOs  who  served  as  board  participants  during  the  field  validation. 
Almost  all  had  prior  experience  as  board  members.  They  commented 
that  there  were  a  lot  of  board  activities  in  the  Army  besides  the  semi- 
centralized  board,  including  E4  boards,  soldier  of  the  month  boards. 
Sergeant  Morales,  and  Audie  Murphy  boards.  Since  little  guidance  exists 
for  preparing  for  and  interacting  with  the  soldier  during  board 
proceedings,  they  felt  that  the  semi-structured  interview  would  be  a 
good  training  vehicle  for  supervisors,  giving  them  specific  practice  in 
how  to  apply  and  interpret  results. 


A  logical  place  to  use  some  of  these 
measures  is  in  NCOES  -  PLDC  and 
BNCOC.  NCOES  is  currently 
undergoing  study  and  evaluation  as 
part  of  the  Army  Transformation  and 
we  will  make  our  findings  available 
to  those  charged  with  this  mission. 


The  Future  of  NCO 
Selection  and  Development 

Significant  changes  to  the  semi-centralized  promotion  system  will  be 
deliberate,  based  on  scientific  evidence  of  improvement,  and  in 
conjunction  with  other  NCO  leadership  initiatives,  coupled  with 
command  and  user  buy-in.  This  effort  is  a  first  step  to  lay  a  foundation 
for  inevitable  change  to  the  system.  It  has  identified  some  viable  tools 
that  could  improve  performance  of  the  NCO  corps  either  through 
changes  to  promotion  requirements  or  through  counseling  or  NCOES 
improvements. 

An  option  that  will  be  evaluated  further  is  to  combine  elements  of 
some  of  the  promising  NC021  measures  (e.g.,  the  LeadEx  and 
Temperament  Indicator)  to  form  a  Leadership  Assessment  Tool  that 
could  be  administered  to  semi-centralized  promotion  candidates.  A 
logical  next  step  is  to  try-out  such  a  concept  under  operational  conditions 
over  a  period  of  time  in  an  experimental  setting  that  could  last  for  1-2 
years.  It  would  be  “experimental”  in  that  soldier’s  promotions  would 
not  be  affected  by  the  new  leadership  measure.  Other  than  that,  it  would 
be  applied  along  side  the  current  promotion  criteria  on  a  wide  basis  to 
soldiers  under  existing  field  conditions.  Work  is  progressing  on 
implementing  such  a  try-out. 
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Your  Input  Counts 


Measuring  performance... 


Now  you  can  provide  feedback  by  giving  us  your  reaction  to  what  we 
have  done  and  what  we  propose  to  do.  Any  comments  are  welcome. 
Send  your  input  to: 


Directorate  of  Military  Personnel  Policy  (DMPP) 
Army  G-1 
DAPE-MPE-PD 
300  Army  Pentagon 
Washington,  DC  20310-0300 
DSN:  224-7182  or  COMM:  703-614-7182 
Or  provide  input  by  email  to 
gerald.purcell@hqda.army.mil 


In  this  study,  we  used  supervisor 
ratings  of  observed  and  expected 
future  performance  as  the  criteria 
for  evaluating  the  predictors. 
Performance  ratings  are  measures 
with  established  effectiveness  and 
are  relatively  easy  to  collect.  But 
they  are  not  the  only  measures  that 
could  be  used.  MOS-specific 
tactical  and  technical  knowledge 
tests  and  hands-on  performance 
tests  of  technical  and  leadership 
skills  would  also  be  valuable 
measures.  The  drawback  to  these 
is  their  difficulty  in  preparation  and 
administration.  However,  we  are 
exploring  ways  that  simulations  can 
be  used  to  provide  standard 
evaluations  and  reduce  the 
overhead  requirements  of  these 
measures.  Further,  ARI  is  currently 
conducting  research  to  develop 
MOS-specific  tactical  and  technical 
competency  tests  that  overcome 
the  preparation,  administration,  and 
operational  obstacles  of  former 
MOS  skill  tests. 


[The  Army  must] . . .  “develop  Enlisted  Personnel  and 
Leader  Development  Systems  that  produce  a 
deployable,  agile,  versatile,  lethal  and  survivable 
soldier  who  is  persuasive  in  peace,  invincible  in 
war,  and  victorious  in  any  form  of  conflict.” 


Excerpt  from  the  Army  Developmental  Systems  XXI  vision  statement 
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